Improved Metrics for Non-Classic Test Prioritization Problems
نویسنده
چکیده
The average percent of faults detected (APFD) and its variant versions are widely used to evaluate prioritized test suite’s efficiency. However, APFD is only available for classic test case prioritization, where all prioritized test suites under comparison contain the same number of test cases. If people overlook this phenomenon, they may obtain incorrect results in some non-classic scenarios, where prioritized test suites have different sizes. In addition, it can’t precisely illustrate the process of fault detection. Besides the APFD, most of its variants have similar problems. This paper points out these limitations in detail, by analyzing the physical explanation of APFD series metrics. To avoid limitations, a series of metrics including RAPFD, RAPFDC , RAPFDW and RAPFDCW are proposed for different types of scenarios. All proposed metrics refer to both the speed of fault detection and the constraint of testing resource. There is an example in this paper showing that proposed metrics provide much more precise illustrations of fault detection process and fault detection efficiency of test suite. Keywords—software testing, test case prioritization, fault detection efficiency, metric
منابع مشابه
Understanding and Measuring the Sources of Variation in the Prioritization of Regression Test Suites
Test case prioritization techniques let testers order their test cases so that those with higher priority, according to some criterion, are executed earlier than those with lower priority. In previous work, we examined a variety of prioritization techniques to determine their ability to improve the rate of fault detection of test suites. Our studies showed that the rate of fault detection of te...
متن کاملPerforming Regression Test Prioritization for Time-Constrained Execution Using a Genetic Algorithm
Test case prioritization techniques organize the test cases in a test suite, allowing for an increase in the effectiveness of testing. One performance goal, the fault-detection rate, is a measure of how quickly faults are detected during the testing process. An improved rate of fault detection can provide faster feedback regarding the quality of the system under test, but frequently, complete t...
متن کاملMulti-objective test case prioritization in highly configurable systems: A case study
Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The order of test cases is determined by prioritization objectives such as covering code or critical components as rapidly as possible. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge nu...
متن کاملOn Increasing System Test Effectiveness through a Test Case Prioritization Model Using Static Metrics and System Failure Data
System testing is the last phase before the product is delivered for customer use and thus represents the last opportunity for verifying that the system functions correctly and as desired by customers. System test is time consuming in that it involves configuring and testing multiple complete, integrated systems (including hardware, operating system, and cooperating and coexisting applications)...
متن کاملA Collaborative Filtering Recommender System for Test Case Prioritization in Web Applications
The use of relevant metrics of software systems could improve various software engineering tasks, but identifying relationships among metrics is not simple and can be very time consuming. Recommender systems can help with this decision-making process; many applications have utilized these systems to improve the performance of their applications. To investigate the potential benefits of recommen...
متن کامل